System for increasing data access in network having compression device for determining and controlling data/object compression based on predetermined maximum percentage of CPU processing capacity

ABSTRACT

An apparatus for increased data access from data of the type including at least one of a file, an object and a directory in a file/object oriented network comprises a compression device having means for determining when processing said CPU reaches a predetermined percentage of maximum processing capacity, and means operably associated with said determining means for controlling compression of data/object upon reaching said predetermined percentage in a manner to aid processing to fall below said predetermined percentage.

This is a continuation-in-part of U.S. Ser. No. 09/441,495 filed Nov.17, 1999, now a U.S. Pat. No. 6,339,787 issued on Jan. 12, 2002, whichis a continuation-in-part of U.S. Ser. No. 08/956,190 filed Oct. 22,1997, now a U.S. Pat. No. 6,012,085 issued on Jan. 4, 2000, which is acontinuation-in-part of U.S. Ser. No. 08/888,311 filed Jul. 3, 1997, nowa U.S. Pat. No. 5,835,943 issued on Nov. 10, 1998, which is acontinuation-in-part of U.S. Ser. No. 08/565,393 filed Nov. 30, 1995,now a U.S. Pat. No. 5,682,514 issued on Oct. 28, 1997.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to data access in a file/object orientednetwork system. More particularly, the present invention is directed toa client-agent-server utility which increases the speed in which data inthe form of files, objects and directories are accessed across slow linkcommunications via remote node caching and provides verification,selective object compression, processor capacity monitoring in order todisable process intensive components such as compression, selectiveprefetch and concatenation of fresh objects and indicators of cachecorrectness.

2. Related Art

There exist operating systems which are equipped to handle caching andverifying as well as compression of data. Traditionally, in a remoteclient's caching system, optimization in retrieving data is limited toprefetching. In other words an application program in a remote clientrequests from a file server transmission of a predetermined number ofbytes of information (e.g., x bytes) and the operating system on theclient prefetches the requested data plus another number of bytes ofinformation (e.g., x+y bytes). Thus, when the application requests thebytes, it already exists in its readily accessible memory (cache).

In addition, there also exist problems with verification of directoriesin existing systems. It has been found, for example, that two remoteclients concurrently accessing data and attempting to verify a directorywill not necessarily obtain the same data due to the fact that the datafrom the file server computer will not necessarily send out the data inthe same order to each of the remote clients. Thus, there is no clearindication whether the directory data is current.

In a desktop caching system, a high speed memory is used to cache datathat is stored on a hard disk. While a desk-top cache program, such asMicrosoft's SmartDrive, is a useful tool to increase performance fromthe random access memory (RAM), this type of caching technique is notapplicable to remote environments because of its inability to correctlyhandle multiple remote clients accessing the same data filesconcurrently, i.e., it is likely to corrupt the data.

File servers have employed caching techniques which parallel techniquesof the desktop. Here, the file server deviates in protecting againstmultiple common data user access by implementing or providing a filelocking service to clients.

Many object oriented network systems include web browsers which commonlymanifest themselves on an object retrieval side of the remote client,such as Netscape's Navigator or as Lotus Notes clients, and include webservers which commonly manifest themselves on the object server side,such as Notes servers, are equipped to maintain a cache of objects toavoid unnecessary retrieval of objects from a network of objectproviders. Cache correctness is determined through a given technique.

Many existing object oriented network systems employ inefficient datacommunication protocols to transfer object updates to replicas of anobject collection. For example, during the replication process thattakes place between a Lotus Notes™ client and server each object updateis requested separately which results in extra packet exchanges andinefficiency.

Existing object oriented network systems often employ aclient-agent-server utility (the “agent”) to further reduce unnecessaryretrieval of objects from a network of object provider. These agents areoften termed as “proxy servers” since they retrieve objects from anetwork of object providers on behalf of a set of clients. In thissituation, the agent maintains a cache of objects and monitors andresponds to object retrieval requests from one or more remote clients.The agent may fulfill the request which emanates from a client byretrieving the object from its cache rather than forwarding the requestto the network of object providers.

As shown in FIG. 1, the related art includes a remote client computerhaving an operating system (OS) with a file system interface (FSI).Operatively connected to the FSI is a local file system (LFS) which inturn is operatively connected to a RAM based disk cacher (RBDC), diskdriver (DD) and permanent storage disk (PSD). The PSD may include objectretrieval application cache (ORAC) and object collection Replicas(OCRs).

Object retrieval applications (ORAs) exist in the remote client whichhave the ability to retrieve objects and to store OCRs into the PSD viathe LFS via the FSI. These OCRs are retrieved through an ObjectRetrival/Storage interface (ORSI) which employs an Object Retriever(OR).

Operatively connected to the FSI is a network file redirector (NFR) withprefetch capability, and a network transport layer (NTL) connected to aWAN driver. Aside from the OS, there exist application programs (AP)which employs the OS via the FSI. A communication server (CS) connectsto the remote client computer and includes a WAN. driver, routing layerand LAN driver. The CS connects through a LAN link to a file servercomputer.

The file/object server computer has an OS. The file/object servercomputer OS includes an NTL connected to a LAN driver and an FSIconnected to LFS which in turn is connected to an RBDC, a DD and a PSD.Aside from the OS, there exists a file/object server application whichemploys the OS via the FSI.

An object proxy server (OPS) may also exist operatively connected to thecommunication server and the file object server. The OPS includes andORSI, and OR, NTL, LAN driver, FSI, RBDC and DD as shown in FIG. 1. TheOPS maintains an object cache for the purpose of maintaining an objectcache on PSD via an FSI. The OPS retrieves objects via an ORSI which isoperatively connected to an Object Retriever (OR).

A further problem associated with these prior systems is their inabilityto provide a remote client user with greater speed of access to objectcollection updates because of inefficient or “chatty” data communicationprotocols. This chattiness usually manifests itself in extra packetexchanges to accomplish the communication of the object collectionupdates by requesting each object update individually. In a satellitebased communication link, latency is an important factor where thesend/receive acknowledgment cycle of even the smallest data unit cantake several seconds to accomplish.

The problem associated with these prior systems is their inability toprovide a remote client user with greater speed of access to file/objectserver data and/or file/object server directories. This is especially sobecause of the type of link in which the remote client may be accessingthe data through, such as a modem phone link. In the context of thepresent invention, “remote client” is defined as a user, accessing dataover a relatively slow link, such as a modem phone link. A typical modemphone link provides a transfer rate of about 28.8 kilobits ofinformation per second. This is contrasted with a link in a LANconnection which can transfer at about 10 Megabits per second. Theseremote clients are thus greatly limited in speed of access.

As described in a parent application, an incoherent data with the thatof the file/object server, employs a compression mechanism fortransmitting the “flesh” block of data to the remote client. Thisprovided performance gains via using a method of concatenating theobject updates into a contiguous stream of data and intelligentlyapplying data compression to the portions of the data stream which wouldbenefit from the compression. The determination of when to apply datacompression is based on weighing the benefits of the data reduction thatwould be achieved and the speed of the communication link versus thetime it would take to perform the data compression, IE the slower thecommunication link and the more data reduction that would be achieved,the more likely that data compression will be applied to the datastream.

While these improvements have aided the speed of access in data over anetwork, there remains a need to improve the speed in which data istransferred. Accordingly, the invention at hand is directed atmaintaining high speed data transfer while protecting the integrity ofoperability of the server computer.

SUMMARY OF THE INVENTION

The present invention improves remote clients access and verification ofobjects and data in files and directories through a file/object orientednetwork environment.

It is an object to maintain optimized speed in which a remote client canaccess data and directories through a server.

It is another object to maintain integrity of the accessed data anddirectory while increasing the speed in which the data is accessed.

Still, another object is to add intelligence to a server computer inorder to reduce the overall time in which a remote client accesses datawhile also maintaining integrity thereof.

Other objects and advantages will be readily apparent from reading thefollowing description and viewing the drawings.

Accordingly, one aspect of the invention is directed to a file/objectserver CPU having which utilizes a data/object compression, whichincludes a compression device having means for determining whenprocessing the CPU reaches a predetermined percentage of maximumprocessing capacity, and means operably associated with the determiningmeans for controllably employing compression of data/object uponreaching said predetermined percentage in a manner to aid processing tofall below said predetermined percentage.

The compression device further includes means for enabling compressionof data/objects upon falling below the predetermined percentage. Thedetermining means is further characterized to monitor the processingcapacity over time, which can be periodic. The determining means ischaracterized to include means for evaluating average process usagepercentage of data/object to be compressed prior to compression.

Terminology

“Caching” is the function of retrieving an object from a relatively highspeed storage device from a list of most-recently-used objects.

“Cache” is a file which resides in permanent storage and contains themost-recently-used blocks of data read from a remote file/object server.

“Compression” can mean front and/or rear compression and can include asingle or number techniques which amounts in condensing the data/objectsin order to render more efficient transfer thereof.

“Data” referred to herein is inclusive of an object, directory and/or afile.

“File/object oriented distributed network,” as used in the presentinvention, will include a network wherein the file/object servercomputer data is accessed via the following set of file system or objectretrieval primitives: OPEN, CREATE, READ, WRITE, SEEK, LOCK, UNLOCK,CLOSE, DIRECTORY REQUEST, GET OBJECT, and SYNCHRONIZE COLLECTIONREPLICATION.

“File” means a collection of related data records treated as a basicunit of storage.

“File/Object Server Computer” is a computer which includes a processorwith its associated memory, an operating system, and a permanent storagememory.

A cached object is considered “stale” if it is found to be incoherentwith the actual object as stored on the file/object server.

A cached object is considered “fresh” if it is found to be coherent withthe actual object as stored on the object server.

A “Handle” is the internal address of a unique data structure thatdescribes characteristics about a file, object, object collection orobject database.

An “Object” is a sequence of data of variable length.

An “Open Method” is an indicator of the actions that a program will takeafter opening a file or object database. The actions may be one or moreof, but not limited to, read-only, write-only, open-for programexecution only, open exclusively, open with the intention of lockingregions prior to update, etc.

“Permanent storage memory,” as used herein, includes, but is not limitedto, disk drive, flash RAM or bubble memory, for example.

“Replication” is the process of exchanging modifications betweenreplicas of a collection of objects.

A “Reverse Channel” is the means by which a response message is sentover the same network layer interface in which a request was received.

A “Sub-object” is a portion of an Object.

A “Validator” is a relatively short stream of data which is returned byan object server along with all object which is to be presented to theobject server for purposes of validating the requestor's object cache.

A “chatty” replication data communication protocol is one where extrapacket exchanges are used to request each object update from a set ofobject collection updates individually.

“Streaming” is the method of concatenating a collection of objects intoa larger object for the purposes of more efficient data communicationsby eliminating the overhead packets and communication latency associatedwith the transfer of objects on an individual basis.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the block diagram configuration of the related art.

FIG. 2 illustrates the block diagram configuration of the presentinvention.

FIG. 2a illustrates a block diagram configuration of another embodimentof the invention wherein the intelligence of the cache verifyingcomputer is resident on the object server computer and the intelligenceof the network/file object cacher is resident in the object requesterapplication.

FIG. 3 illustrates a flow chart of the operations of the presentinvention corresponding to the requests within a remote client.

FIG. 4 illustrates a flow chart of the operations of the presentinvention corresponding to OPEN/CREATE requests on remote clientcomputer.

FIG. 4a illustrates a flow chart of the operations of the presentinvention corresponding to a part of the operations in FIG. 4.

FIG. 5 illustrates a flow chart of the operations of the presentinvention corresponding to OPEN/CREATE requests on cache verifyingcomputer.

FIG. 6 illustrates a flow chart of the operations of the presentinvention corresponding to READ requests on remote client computer.

FIG. 7 illustrates a flow chart of the operations of the presentinvention corresponding to READ requests on cache verifying computer.

FIG. 8 illustrates a flow chart of additional operations of the presentinvention corresponding to READ requests in the cache verifyingcomputer.

FIG. 9 illustrates a flow chart of the operations of the presentinvention corresponding to WRITE requests on remote client computer.

FIG. 10 illustrates a flow chart of the operations of the presentinvention corresponding to WRITE requests on cache verifying computer.

FIG. 11 illustrates a flow chart of the operations of the presentinvention corresponding to LOCK requests on remote client computer.

FIG. 12 illustrates a flow chart of the operations of the presentinvention corresponding to LOCK requests on cache verifying computer.

FIG. 13 illustrates a flow chart of the operations of the presentinvention corresponding to CLOSE requests on remote client computer.

FIG. 14 illustrates a flow chart of the operations of the presentinvention corresponding to CLOSE requests on cache verifying computer.

FIG. 15 illustrates a flow chart of the operations of the presentinvention corresponding to DIRECTORY REQUEST on cache verifyingcomputer.

FIG. 16 illustrates a flow chart of the operations of the presentinvention corresponding to a part of the operations in FIG. 15.

FIG. 17 illustrates a flow chart of the operations of the presentinvention corresponding to GET-OBJECT REQUEST on the remote clientcomputer.

FIG. 18 illustrates a flow chart of the operations of the presentinvention corresponding to GET-OBJECT REQUEST on the cache verifyingcomputer.

FIG. 19 illustrates a flow chart of the operations of the presentinvention corresponding to a object cache evaluator.

FIG. 20 illustrates a flow chart of the operations of the presentinvention corresponding to a REPLICATE-SYNCHRONIZATION request on theremote client computer.

FIG. 21 illustrates a flow chart of the operations of the presentinvention Corresponding to a REPLICATE-SYNCHRONIZATION request on thecache verifying computer.

FIG. 22 illustrates a flow chart of the operations of the presentinvention corresponding to a compression/decompression unit.

FIG. 23 illustrates a flow chart of the operations of the presentinvention corresponding to a compression device of the invention whichenables/disables the compression/decompression unit.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

In the description which follows, the representation of the presentinvention is in part presented in terms of program operations executedon a file/object oriented distributed network of computers, but may aswell be applicable to distributed file/object oriented network systems.The operations are steps leading to a certain result. Typically, thesesteps take the form of electrical signals which are manipulated, stored,transmitted, combined, compared or otherwise operated upon by aparticular computer in the network. For simplicity, these signals may bereferred to herein as bits, bytes or data.

The following description describes solutions to the problems associatedwith a remote client computer's ability to access specified data from afile, an object or directory of a file/object server computer located ona network or world wide web. An apparatus and method are disclosed whichpermit the remote client computer to reduce the time for accessing suchdata using a cache verifying computer coupled with a caching technique.

The performance gains realized by the present invention are derived fromthe fact that remote clients tend to repetitively access the same databy performing file reads or object retrievals. If a copy of the data canbe stored in the permanent storage memory of the remote client computerand also verified to be current when it is subsequently retrieved, thiswill improve performance significantly. This is because it requires insuch less bandwidth to verify a block of data than it would to actuallytransfer a block of data. Furthermore when a block of cached data isdeemed to be incoherent with the that of the file/object server, thepresent invention employs a compression mechanism for transmitting the“fresh” block of data to the remote client.

The performance gains realized by the present invention are furtherderived from the fact that existing inefficient methods of transferringobject replica updates with a “chatty” replication communicationprotocol are replaced with a method of concatenating the object updatesinto a contiguous stream of data and intelligently applying datacompression to the portions of the data stream which would benefit fromthe compression. The determination of when to apply data compression isbased on weighing the benefits of the data reduction that would beachieved and the speed of the communication link versus the time itwould take to perform the data compression, IE the slower thecommunication link and the more data reduction that would be achieved,the more likely that data compression will be applied to the datastream.

Referring now to the FIGS. 2-23, the present invention is a networkcomputer system 10 having at least one remote client computer 12, cacheverifying computer 14, communication server 16 and file/object servercomputer 18. The cache verifying computer 14 and file/object servercomputer 18 are connected via a local area network (LAN) link 20. Thecommunication server 16 links the remote client computer 12 to the LAN20, which in turn permits communication with the cache verifyingcomputer 14 and the file/object server compute 18.

The remote client computer 12 communicates via communication link 22 tothe communication server 16. The communication server 16 can be of atype such as that provided by Cisco, 3Com, Shiva, etc., which will actas a router of traffic between the LAN 20 and communication link 22 andconvert data through the LAN 20. The LAN 20 can be Ethernet or TokenRing, for example.

The remote client computer 12 has an operating system (OS) 24 with afile system interface (FSI) 26. Operatively connected to the FSI 26 is alocal file system (LFS) 28 which in turn is operatively connected to aRAM based disk cacher (RBDC) 30, disk driver (DD) 32 and permanentstorage disk (PSD) 34. The PSD 34 includes object retrieval applicationcache (ORAC) 34 a and object collection replicas (OCRs) 34 b. A networkfile redirector (NFR) 36 with prefetch data 37, operatively connects toa network transport layer (NTL) 38 which in turn is connected to a WANdriver 40.

A network file/object cacher (NFOC) 42 is operably disposed between andinterconnects the FSI 26 and NFR 36. The NFOC 42 has operativelyassociated therewith a directory cacher (DC) 43 and directory signaturecomparator (DSC) 49.

The NTL 38 operatively connects to the NFOC 42. Also, the NFOC 42operatively connects to the LFS 28. The NFOC 42 includes a blocksignature generator (BSG) 44 and hit ratio analyzer (HRA) 45, which willbe more fully described hereinafter. Aside from the OS 24, there existson the remote client computer 12 application programs (AP) 46 whichemploy the OS 24 via FSI 26.

The remote client computer 12 also has object retrieval applications(ORAs) 31 which are operatively connected to an object retrieval/storageinterface (ORSI) 27 which is in turn is operatively connected to theobject retriever (OR) 39 via the NFOC 42. The OR 39 operatively connectsto NTL 28. The NFR 36 operatively connects to a file signature retriever(FSR) 33, wherein the file signature may include the time that the filewas last modified, for example. Likewise, the OR 39 operatively connectsto an object signature retriever (OSR) 35. The NFOC 42 also hasoperatively associated therewith a local comparator (LC) 13, acompressor/decompressor (CD) 47, a caching filter (CF) 50, a replicationsynchronizer (RS) 41 and an object cache evaluator (OCE) 29. Each saidOCE are exemplified in the packet formats set forth hereinafter.

The communication server (CS) 16 includes a WAN driver 48, a LAN driver50 and routing layer (RL) 52 operatively interconnecting the WAN driver48 and the LAN 50 driver.

The cache verifying computer 14 includes a cache verifying agent (CVA)54 having a BSG 56 (of the type described herein), a directory signaturegenerator (DSG) 57 and a comparator 58. The CVA 54 also includesoperatively associated therewith CD 51 an object cacher (OC) 59, areplication analyzer (RA) 53 and an associated object retriever (AOR)55, wherein the replication analyzer RA 53 is located on the remoteclient RC 12 and synchronizer 41 is located on the verifier computer 14.The function of the replication analyzer is to determine a list ofdocument updates that should be moved in the data transfer, this isoften accomplished by analyzing a list of document updates supplied byORA (31).

While it is shown that these functional groups are in the CVA, it iscontemplated that these functions can be embedded into the file objectserver. Likewise the function of the NFOC could be embedded into theORA. This is shown in FIG. 2A. It is important that the techniquesdescribed herein be applied only when it would benefit the efficiency ofthe overall system. The most important consideration in this applicationis the speed of the communication link between the remote clientcomputer 12 and the tile/object server computer 18. For example applyingdata compression to an object prior to transmission should only be doneif the reduction of characters transmission from that compressionfunction results in improved response time to the remote client 12; on afast link, the CPU cycles required to perform the compression mayactually slow down the overall responsiveness of the system andtherefore data compression should not be applied in this scenario. Aswill be more apparent in FIG. 23 described hereinafter, the compressiondevice employed herein monitors the processor usage to determine whetherto disable/enable compression of data/objects.

Also, included is an OS 60 having an FSI 62 operatively connected to CVA54, an NFR 64 operatively connected to the FSI 62, an NTL 66 operativelyconnected to the NFR 64 and CVA 54, and a LAN driver 68 operativelyconnected to the NTL 66. The CVA 54 is also operatively connected to anORSI 65 which is in turn operatively connected to an OR 61 and in turnto OSR 63. The OR 61 is operatively connected to the NTL 66.

The file/object server computer 18 includes an OS 70 having a filesystem/object interface (FSOI) 72 which is operatively connected to alocal file system/object database (LFOS) 74 which in turn is connectedto an RBDC 76, a DD 78 and a PSD 80. The OS 70 includes an NTL 82operatively connected to a LAN driver 84. A file/object serverapplication (FOSA) 86 exists on the computer 18 which is operablyconnected to both the NTL 82 and FSI 72. The FOSA 86 includesoperatively associated therewith an object signature (ObS) 87.

It should be noted that one skilled in the art can modify the basicnetwork computer to accomplish the objects set forth herein and thatsuch modifications are believed to fall within the scope of the claimsappended hereto. Alternatively, for example, the cache verifying agent54 could reside as part of the communication server 16 or as a standalone processor with its own memory and operating system. Still, otherpersons skilled in the art will appreciate the verifying agent can beimplemented in other manners to accomplish the goals set forth herein.

The operation of the system is as follows and as represented in FIGS.3-22. The operations discussed hereafter assumes connections have beenmade among all computers 12, 14 and 18 and communication server 16.

On the remote client computer 12. AP 46 makes requests from afile/object server computer 18 wherein the NFOC 42 will intercept a filesystem call or object retrieval call 100 from the AP 46 or ORA 31 andquery whether the data to be acted upon is “remotely located?” 102. Ifthe answer is no, the NFC 42 “instructs” 104 the LFS 28 to handle thedata request. If yes, the type of request is ascertained and handled asfollows.

In the case of OPEN or CREATE 106 requests, the NFOC 42 follows theoperation under 200. The NFOC 42 “invokes” 202 the NFR 36 to process therequest. The NFR 36 asks “whether there is a good status” 204 for therequest. If no, NFR 36 “returns” 205 the results of the operation to therespective AP 46 or ORA 31.

If yes, NFOC 42 “invokes” 207 CF 15 to “ask” if the data, e.g., objectsassociated with a file, should be cached based on a set of pre-definedfiltering rules and goes to operation 160. CF 15 “assesses” 160 how muchtime it takes to access data, e.g., an object. CF 15 “determines” 162“is LAN access available?” If yes, CF 15 “applies” 164 LAN filteringrules. If no, CF 15 “uses” 166 “WAN filtering rules?” From operations164 and 166 follows operation 168 wherein CF 15 “matches” 168file/object name against “never cache” set. The “never cache” set is aset of data never to be cached. CF 15 “asks” 170 “is there a match?” Ifyes, CF 15 “returns” 172 a signal that data should not be cached. If nomatch, CF 15 “matches” 174 file/object name against LC 13 set. CF 15“asks” 176 “should” LC 13 be used?” If yes, CF 15 “returns” 178 a signalthat LC 13 be employed for the subsequent reads and operation 211follows. If no, CF 15 “returns” 180 a signal that CVA's 54 comparator 58be used on subsequent reads and CF 15 “returns” 209 the data to NFR 36at operation 205.

If yes, CF 15 triggers NFOC 42 to examine the characteristics of theopen method and “determine” 211 if the objects associated with this filemay be verified with a LC 13. If yes, then NFOC 42 “obtains” 213 thesignature by employing either the FSR 33 or OSR 35. Then NFOC 42 “asks”215 is the data fresh? meaning, for example, “are the objects associatedwith this file coherent based on the FSR 33. If yes, the NFOC 42 “marks”221 a handle such that all subsequent read requests which can besatisfied from the cache are deemed coherent based on the LC 13. Theresults are returned to operation 205.

If no, then all blocks associated with this data, e.g., file, are“removed” 219 from the cache by the NFOC 42 via LFS 28 and the NFR 36returns 205 the results of the operation to AP 46, for this example.

If “no” was the determination of operation 211, the NFR 36 assigns ahandle to the data and the NFOC 42 “builds and sends” 206 an OPEN/CREATErequest to CVA 54 via NTL 38 which triggers operation 250.

CVA 54 “opens” 252 a file specified in OPEN/CREATE request via FSI 62,NFR 62 and NTL 66. The CVA 54 asks “whether there is a good status onthe file?” 254. If the answer is no, CVA 54 “sends” 256 the bad responseback to NFOC 42 in a reverse channel. If the answer is yes, CVA 54“assigns a handle to the object” 258 and “sends” 260 a good response viaa reverse channel.

NFOC 42 via NTL 38 “receives the response” 208 from CVA 54 and “asks fora good status?” 210. If the answer is no, the NFOC 42 “returns theresults of the original OPEN/CREATE request” 216 to AP 46. If the answeris yes, then the NFOC 42 “associates 212 the handle assigned by the CVA54 with the handle returned by the NFR 36 in operation 202. The NFOC 42“updates” 214 the network file cache via LFS 28 and “returns the resultsobtained by NFR 36” 216 to AP 46 via FSI 26.

In the case of a READ 108 request, the computer 12 follows the operation300. Via the FSI 26 and LFS 28, the NFOC 42 “determines if the requesteddata is in cache?” 302. If the answer is no, a subquery becomes “is thedata locked?” 304. To this subquery, if the answer is no, the NFOC 42“retrieves” 306 the data via NTL 38 from the file/object server computer18 and the NFOC 42 “updates” 308 the network file cache via LFS 28. Ifthe answer to the subquery is yes, the NFOC 42 via the NTL 38 “sends”310 a READ request to CVA 54 which triggers 380. CVA 54 via the FSI 62“reads” 382 the data from the file server computer 18. The CVA 54“sends” 384 a response back to NFOC 42, wherein the data is “received”312 and “updated” 308 as described above. The retrieved data is“returned” 314 by the NFOC 42 to AP 46.

If the data is in cache, NFOC 42 “asks” 301 should the verify be doneusing the LC 13. The answer to this was established based on whether ornot 221 had been invoked during the OPEN operation. If the answer isyes, then NFOC 42 “invokes” 213 LC 13.

If the answer is no, NFOC 42 is triggered to “invoke” 316 the BSG 44 togenerate a signature of data. NFOC 42 via NFR 36 and NTL 38 “sends” 320a VERIFY request having the first signature of data therein to CVA 54which triggers 350.

CVA 54 via FSI 62 “reads” 352 data from the file server computer 18. CVA54 “invokes” 354 BSG 56 to generate a second signature of data. CVA 54“invokes” 356 comparator 58 to compare the first and second signaturesof data and “asks whether there is a match?” 358. If the answer is no,CVA 54 “asks if data is locked?.” 360. If no, the CVA 54 “sends” 362back a bad response to NFOC 42 via a reverse channel. If yes, CVA 54“sends”364 back a bad response to NFOC 42 along with read data via areverse channel. If there is a match of the signatures, CVA 54 “sends”366 a good response back to NFOC 42 via NTL 66.

The NFOC 42 receives 322 the response from CVA 54 and asks “is the datavalid?” 324. If no, NFOC 42 asks “is the data locked?” 326. If notlocked, the NFOC 42 retrieves data 306 as described above. If locked,data will have been “returned” 328 for updating( per 308. If the datawas valid, NFOC 42 returns the data to AP 46.

In the case of a WRITE 110 request, the computer 12 follows theoperation 400. The NFOC 42 “asks is the data locked?” 402. If no, theNFR 36 is invoked to “write” 404 to the file server computer 18. If thedata is locked, NFOC 42 via NTL 38 “sends” 406 a WRITE request to CVA 54which triggers 450. CVA 54 “writes” 452 data to file server computer 18via FSI 62. CVA 54 “sends” 454 back a response to NFOC 42 which“receives” 408 the response. The NFOC 42 “asks is the data in cache?”410. If no, LFS 28 “reports status” 412 to AP 46. If yes, NFOC 42“updates” 414 network cache via LFS 28 and “reports status” 412 to AP46.

In the case of LOCK/UNLOCK 112 request, operation 500 is employed. TheNFOC 42 “builds” 502 an LOCK/UNLOCK request. The NFOC 42 via NTL 38“sends” 504 the LOCK/UNLOCK request to CVA 54 which triggers operation550. CVA 54 “sends” 552 an LOCK/UNLOCK request to the file servercomputer 18 via FSI 62. CVA 54 “sends” 554 a response back to NFOC 42via a reverse channel. The NFOC 42 “receives” 506 the response and“returns” 508 the results to AP 46.

In the case of a CLOSE 114 request, operation 600 is employed. The NFOC42 “builds” 602 a CLOSE request. The NFOC 42 via NTL 38 “sends” 604 theCLOSE request to CVA 54 which triggers operation 650. CVA 54“performs's” 652 internal processing of the request. CVA 54 “sends” 654a response back to NFOC 42. The NFOC 42 “receives” 606 the response andinvokes the NFR 36 to “close” 608 the file and “return” 610 the resultsto AP 46.

In the case of a DIRECTORY REQUEST 115, operation 700 is employed. Here,the NFOC 42 “processes” 701 a first directory sub-object request.

If the sub-object is not a first, NFOC 42 “retrieves” 703 the nextdirectory sub-object from cache via LFS 28. NFOC 42 “asks” 704 whetherthis is the last sub-object from cache via LFS 28? If no, NFC “returns”705 a sub-object to AP 46. If yes and it is the last sub-object, NFOC 42“returns” 706 a “no more objects” status to AP 46.

If the sub-object is the first directory sub-object, the NFOC 42“determines” if the requested object is in cache 702. If no, the NFOC 42“sends” 710 a directory verify request to CVA 54 via NTL 38. Thistriggers the steps 750 and NFOC 42 waits to “receive” 711 signature fromCVA 54. As seen in FIG. 16, the steps 750 are preformed by the CVA 54.Particularly, the DSG 57 “initializes” 751 signature of a directory. TheDSG 57 “retrieves”752 the first directory sub-object from the FS 18 viaNTL 66. The DSG 57 “asks” 753 is this the last sub-object? If no, DSG 57“factors” 754 the signature of this sub-object into the overallsignature of the directory. The DSG 57 then “retrieves” 755 the nextsub-object from FS 18 and returns to step 753. If the last sub-object,CVA 54 “sends” 756 back signature of directory to NFOC 42 at block 724and proceeds therefrom.

If yes and in cache, the NFOC 42 “retrieves” 719 signature associatedwith this directory request from cache via LFS 28. NFOC 42 “sends” 720directory verify request to CVA 54 via NTL 38. This triggers the steps750 wherein NFOC 42 waits and “receives” 721 signature from CVA 54. NFOC42 “invokes” 722 DSC 46 to compare whether signature matches theretrieved signature in 719? If yes and the signatures match, NFOC 42“returns”723 the first sub-object from cache via LFS 28 and returns itto AP 46. If no and the signature does not match, NFOC 42 “invokes” 724NFR 36 to retrieve the first directory sub-object. NFOC 42 “stores” 725the sub-object into cache via LFS 28. NFOC 42 “asks” 726 whether this isthe last sub-object? It no and it is not the last sub-object, NFOC 42invokes NFR 36 to “retrieve” the next directory sub-object and returnsto step 725. If yes and it is the last sub-object, NFOC 42 “stores” 728the signature obtained via 721 or 711 into cache via LFS 28. NFOC 42“returns” 729 first sub-object from cache via LFS 28 and returns thesame to AP 46.

In the case of a GET-OBJECT REQUEST 116 operation 800 is employed. TheNFOC 42 invokes OCE 29 to perform operation 980 to determine objectstatus. OCE 29 “asks” 982 “does object exist in cache? If no, OCE 29“returns” 990 a signal that object is not in cache. If yes, OCE 29“asks” 984 “is there an object expiration tag?”

If yes and there is a object expriation tag, OCE 29 “asks” 986 “hasobject expiration tag expired?” If yes, OCE “returns” 996 a digitalsignature and signal that object is stale. If no, OCE 29 “returns” 988signal that cached object is fresh.

If no and there is a object expiration tag, OCE 29 “asks” 992 “is therea validator in the object?” If yes, OCE 29 “returns” 994 a signal thatobject should be validated using validator. If no, OCE 29 “returns” 998a signal that object should be validated using digital signature.

Once object status has been determined via 802, NFOC 42 “asks” 358 “doesobject need to be retrieved or verified?” If yes, NFOC 42 “sends” 806 aGet-Object request packet to CVA 54. This triggers operation 850 andNFOC 42 waits for response from CVA 54.

The operations 850 et seq. are as follows. NFOC 42 “gets” 850 an objecthandler on CVA 54. CVA 54 “asks” 852 is there a fresh object in theobject cache? If yes, CVA 54 “invokes” 854 signature comparator 58. CVA54 “asks” 856 is the signature valid? If yes, CVA 54 “determines” 858 ifthere are associated objects. If no, CVA 54 “appends” 868 the cachedobject into a signal response to be sent back to NFOC 42 and thenreturns to the operations 858 and those following.

If the answer to operation 852 is no, the CVA 54 “retrieves” 864 anobject from FOS 18 via ORSl 65. CVA 54 “invokes” 866 CD 51 and thenstores the object into cache via OC 59 and goes to operations 868 andthose which follow.

From operation 858, CVA 54 “asks” 860 are there associated objects? Ifyes, CVA “appends” 870 freshness indicator to a response signal for eachassociated object that exists in cache. CVA 54 then “sends” 862 aresponse signal back to NFOC 42 via NTL. 66. If the answer to operation860 was no, the operation 862 follows.

NFOC 42 “asks” 810 “is the object fresh?” If no, NFOC 42 “stores” 812fresh object into object cache via LFS 28. If yes, NFOC 42 “asks” 816“are there associated object freshness indicators?” If yes, NFOC 42“marks” 818 associated objects as fresh in the object cache via LFS 28.If no, NFOC 42 invokes CD 47 and “returns” 820 the object to ORA 31. Ifthe answer to operation 358 is no, the NFOC 42 invokes CD 47 and“returns” 820 the object to ORA 31.

In the case of a REPLICATION-SYNCHRONIZE REQUEST operation 900 isemployed. NFOC 42 “builds” 902 a Replication-Synchronize request and“sends” the request to CVA 54 which triggers operation 950.

CVA 54 “determines” 952 a set of objects to send to NFOC 42 based uponthe Replication-Synchronize request. CVA 54 “sets” 954 “x” =0 as indexinto an object set. CVA 54 “asks” 956 “is the object in cache andfresh?” If no, CVA 54 “retrieves” 966 the object via FOSI 62.

At this point, the CVA 54 (or FOS 18) deploys 1980 a Compression BackoffMonitor (CBM) 151 as depicted in FIG. 23. The CBM 151 determines 1982average processor usage percentage for same or higher priority tasksover a prior predetermined number (x) of seconds. CBM 151 determines1984 whether the average processor usage is greater the a predeterminedpercentage process capacity (y %).

If No, CBM 151 determines 1986 whether CVA 54, for example, is currentlyaccepting connections. CVA 54 modifies 1988 status in Verifier Databaseto Accepting Connections and sleeps 1990 for a preset number of seconds(z).

If yes to step 1986 CVA 54, for example, determines 1992 whether CVA 54,for example, is currently accepting connections. If No, CVA 54 sleeps1990 for a preset number of seconds (z). If yes, CVA 54 modifies 1994status in Verifier Database to Not Accepting Connections. CVA 54 setscompression level for existing connections 1995 down to the lowestlevel. CVA 54 returns 1996 a busy signal response to subsequentconnection requests. The steps relating to CBM 151 are inserted prior toinvoking CD 51.

Once in accepting mode from the CBM 151, CVA 54 “invokes” 1964 CD 51 tocompress based on objective characteristics. CVA 54 “stores” 962 objects(x) into cache. CVA 54 “asks” 958 “is the object in cache and fresh?” Ifno, CVA 54 “sets” 960 “x=x+1” and returns to operation 956. If theanswer to operation 956 and 958 is yes, CVA 54 “streams”970 compressedset of fresh objects back to NFOC 42 via NTL.

NFOC 42 “obtains” 904 stream of fresh objects response from CVA 54. NFOC42 “invokes” CD 47 and updates object collection replica via ORSI 27.NFOC 42 “sends” 904 a signal back to ORA 31 that replication iscomplete.

When CD 47 or CD 51 (either referred to as CD in this paragraph) isinvoked, the operations under 920 are performed. CD “asks” 922 “is thisa decompress request?” If no, CD “asks” 924 “can it be determined thatnegligible benefits will result from compression using availabletechniques? If no, CD “asks” 926 “can an appropriate compression methodbe determined?” If no, CD “selects” 928 a default compression method. CD“asks” 936 “did a sample of the default compression method yield goodresults? In this embodiment, results are deemed to be “good” if theapplication of the compression algorithm would result in a fasteroverall process. For example the CPU time required to perform thecompression may be greater then the time it takes to transfer theadditional bytes of an uncompressed data object if the speed of thecommunication link is very fast. The CD takes into consideration thespeed of the communication link when determining if the results of theobject sample yielded “good” results. If no, CD “selects” 932compression method as none. If the results of 936 are good, CD “emits”938 a stream of uncompressed data to one of permanent disk storage andto an in-memory buffer depending on invoker's request parameters. If theanswer to operation 924 was yes, then operations 932 and 938 arefollowed. If the answer to operation 926 is yes, CD “selects” 934 thecompression method that is appropriate for the object type and operation938 follows. If the answer to operation 922 is yes, CD “selects” 930 adecompression method that corresponds to the compression method used. CD“emits” 940 a stream of uncompressed data to one of permanent diskstorage and to an in-memory buffer depending on invoker's requestparameters.

Though not shown in FIGS. 900 and 950, it is contemplated that similarmethods arc employed for “pushing” object updates from the remote clientvia NFOC 42 to the Server Computer 18 when processing theREPLICATION-SYNCRONIZE request. To apply the methods to the “push” ofobject updates, NFOC 42 employs a replication analyzer 31′ to format theobject updates into a contiguous “stream” applying compression whenappropriate. Similarly, the Server 18 employs a replication synchronizer61′ to process the object updates from the stream and apply the updatesto the local object collection. This embodiment is depicted in FIG. 2a,wherein operative elements of the former CVA 54 have been incorporatedinto the Server 18.

By way of example, the following packet formats define this clientserver protocol:

// // TYPE DEFINITIONS // BYTE =>an 8 bit value (octet) unsigned DWORD=> a 16 bit value in which network byte ordering is not important WORD=> 32 bit value in which network byte ordering is not important MDWORD=> 32 bit value in which network byte ordering is important andrepresented using “motorola or big endian” format // START CONNECTIONREQUEST typedef struct { BYTE bFunctionCode; // always 0x00 BYTE bResv;// WORD wSequenceValue; // } CVP_START_CONNECTION_REQ,*pCVP_START_CONNECTION_REQ; // START CONNECTION RESPONSE typedef struct{ BYTE bFunctionCode; // always 0x80 BYTE bStatus;// WORD wSequenceValue// Same value as in request DWORD dConnectionHandle; // }CVP_START_CONNECTION_RSP, *pCVP_START_CONNECTION_RSP; // END CONNECTIONREQUEST typedef struct } BYTE bFunctioncode; // always 0x01 BYTE bResv;// WORD wSequencevalue; // DWORD dConnectionHandle; // // as returned onstart connection } CVP_END_CONNECTION_REQ, *CVP_END_CONNECTION_REQ; //END CONNECTION RESPONSE typedef struct { BYTE bFunctionCode; // always0x81 BYTE bStatus; // WORD wSequencevalue; // Same value as in request }CVP_END_CONNECTION_RSP, *pCVP_END_CONNECTION_RSP; // OPEN OR CREATE FILEREQUEST typedef struct { BYTE bfunctioncode; // always 0x02 BYTE bResv;// WORD wSequenceValue; // DWORD dConnectionHandle; // As returned onSTART_CONNECT MDWORD dFileAttributesMask; // BYTE zFilePath[512]; //nullterminated file name } CVP_OPEN_OR_CREATE_FILE_REQ,*pCVP_OPEN_OR_CREATE_FILE_REQ; // OPEN OR OR CREATE FILE RESPONSEtypedef struct { BYTE bFunctionCode; // always 0x82 BYTE bStatus; //WORD wSequenceValue; // Same value as in request DWORDdVerifiersFileHandle; // } CVP_OPEN_OR_CREATE_FILE_RSP,*pCVP_OPEN_OR_CREATE_FILE_RSP; // CLOSE FILE REQUEST typedef struct {BYTE bFunctionCode; // always 0x03 BYTE bResv; // WORD wSequenceValue;// DWORD dConnectionHandle; // As returned on START_CONNECT DWORDdVerifiersFileHandle; // As returned on OPEN_OR_CREATE }CVP_CLOSE_FILE_REQ *pCVP_CLOSE_FILE_REQ; // CLOSE FILE RESPONSE typedefstruct { BYTE bFunctionCode; // always 0x83 BYTE bStatus; // WORDwSequenceValue; // Same value as in request } CVP_CLOSE_FILE_RSP,*pCVP_CLOSE_FILE_RSP; // LOCK REGION REQUEST typedef struct { BYTEbFunctionCode; // always 0x04 BYTE bResv; // WORD wSequenceValue; //DWORD dConnectionHandle; //As returned on START_CONNECT DWORDdVerifiersFileHandle; // As returned on OPEN_OR_CREATE MDWORDdSeekValue; //offset into file MDWORD dLength; //number of bytes to lock} CVP_LOCK_REGION_REQ, *pCVP_LOCK_REGION_REQ; // LOCK REGION RESPONSEtypedef struct { BYTE bFunctionCode; // always 0x84 BYTE bStatus; //WORD wSequenceValue; // Same value as in requestDWORD  dVerifiersLockHandle; } CVP_LOCK_REGION_RSP,*pCVP_LOCK_REGION_RSP; // UNLOCK REGION REQUEST typedef struct { BYTEbFunctionCode; // always 0x05 BYTE bResv; // WORD wSequenceValue; //DWORD dConnectionHandle;//As returned on START_CONNECT DWORDdVerifiersLockHandle; // As returned LOCK REGION }CVP_UNLOCK_REGION_REQ, *pCVP_UNLOCK_REGION_REQ; // UNLOCK REGIONRESPONSE typedef struct { BYTE bFunctionCode; // always 0x85 BYTEbStatus, WORD wSequenceValue; // Same value as in request }CVP_UNLOCK_REGION_RSP. *pCVP_UNLOCK_REGION_RSP; // VERIFY REGION REQUESTtypedef struct { BYTE bFunctionCode; // always 0x06 BYTE bResv; // ifstatus is not 0xF1 WORD wSequenceValue; // DWORD dConnectionHandle; //As returned on START_CONNECT DWORD dVerifiersFileHandle; // As returnedon OPEN_OR_CREATE MDWORD dSeekValue; // offset into file MDWORD dLength;//number of bytes to verify BYTE Signature[8]; //CRC adaptation }CVP_VERIFY_REGION_REQ, *pCVP VERIFY_REGION_REQ; // VERIFY REGIONRESPONSE # 1 (not locked data) typedef struct { BYTE bFunctionCode; //always 0x86 BYTE bStatus; // WORD wSequenceValue; // Same value as inrequest } CVP_VERIFY_REGION_RSP, *pCVP_VERIFY_REGION_RSP; // // VERIFYREGION RESPONSE #2 // (if signatures did not match and region waslocked) // typedef struct { BYTE bFunctionCode; // always 0x86 BYTEbStatus; //status = 0xF1 for this case WORD wSequenceValue; // Samevalue as in request MDWORD dLength; // # of bytes that follow charTheData[];  // } CVP_VERIFY_REGION_RSP, #pCVP_VERIFY_REGION_RSP; // READREGION REQUEST // (sent only when reading from a locked region) typedefstruct { BYTE bFunctionCode; // always 0x07 BYTE bResv;  // WORDwSequenceValue; // DWORD dConnectionHandle; // As returned onSTART_CONNECT DWORD dVerifiersFileHandle; // As returned onOPEN_OR_CREATE MDWORD dSeekValue; //offset into file WDWORD dLength; //number of bytes to read } CVP_READ_REGION_REQ, *pCVP_READ_REGION_REQ; //// READ REGION RESPONSE // typedef struct { BYTE bFunctionCode; //always0x87 BYTE bStatus; //status 0xF1 for this case WORD wSequenceValue; //Same value as in request MDWORD dLength; // # of bytes that follow cbarTheData[]; // } CVP_READ_REGION_RSP, *pCVP_READ REGION RSP; // // WRITEREGION REQUEST // (sent only for when writing to a looked region) //typedef struct { BYTE bFunctionCode; // always 0x08 BYTE bResv; // WORDwSequenceValue; // DWORD dConnectionHandle; // As returned onSTART_CONNECT DWORD dVerifiersFileHandle: // As returned onOPEN_OR_CREATE MDWORD dSeekValue; // offset into file MDWORD dLength; //number of bytes to write BYTE TheData []; // data to be written }CVP_WRITE_REGION_REQ, *pCVP_WRITE_REGION_REQ; // WRITE REGION RESPONSEtypedef struct { BYTE bFunctionCode; // always 0x88 BYTE bStatus;//status WORD wSequenceValue; // Same value as in request DWORD dLength;// # of bytes written } CVP_WRITE_REGION_RSP, *pCVP_WR1TE_REGION_RSP; //// VERIFY DIRECTORY REQUEST // typedef struct { BYTE  bFunctionCode; //always 0x0A BYTE  bResv; // WORD  wSequenceValue;  // DWORDdConnectionHandle;  // As returned on START_CONNECT MDWORDwFilesInDirectory Count; // MDWORD dAttributeMask;  // 0x00000001 = readonly file // 0x00000002 hidden file // 0x00000004 = system file //0x00000008 = volume label // 0x00000010 = directory // 0x00000020 =changed and not archived BYTE Signature[8];  //  BYTE zSearchPath[512];//null terminated file name } CVP_VERIFY_DIRECTORY_REQ,*pCVP_VERIFY_DIRECTORY_REQ; // VERIFY DIRECTORY RESPONSE RESPONSEtypedef struct { BYTE  bFunctionCode; // always 0x8A BYTE  bStatus; //WORD  wSequenceValue; // Same value as in request BYTE Signature[8];  // } CVP_VERIFY_DIRECTORY_RSP,*pCVP_VERIFY_DIRECTORY_RSP; // // GET OBJECT REQUEST HEADER //typedefstruct { BYTE  bFunctionCHSe; // always 0x48 BYTE  bResv; //DWORD  dMessageLen; DWORD  wSequenceValue;  // DWORD  dConnectionHandle;// DWORD dVerifiersDbHandle; // 0 for Web Objects #define HAPPROT_HTTP 1// 1 = HTTP type get request #define HAPPROT_LOTUSNOTES 2 // 2 = LotusNotes type Get request  BYTE  *bProtocol; BYTE  bCommandBlockCount; //Number of GetObject command blocks which follow } HAP_GET_OBJECT_REQHDR,*pHAP_GET_OBJECT_REQHDR; // // GET OBJECT REQUEST // typedef struct {HAP_GET_OBJECT_REQHDR Hdr; // fixed header BYTE  FirstCommandBlock[1];BYTE  sMd5Signature[16]; // if packet signatures required }HAP_GET_OBJECT_REQ, *pHAP_GET_OBJECT_REQ; // // GET OBJECT COMMAND BLKHEADER // typedef struct { MDWORD dLength; // Length of command block#define GETOBJCMD_F_GETOBJECT 0x01 // 1=Get Object #defineGETOBJCMD_F_GETOBJECT EMB 0x02 // 2=Get Object and Embedded Objects#define GETOBJCMD_F_VEROBJECT 0x04 // 4=Verify Object //if 0x01 also onthe retrieve object #define GETOBJCMD_F_GETDIROBJECT 0x08 // 8=Getdirectory object #define GETOBJCMD_F_VERDIROBJECT 0x10 // 10=Verify DirObject // if 0x08 also on the retrieve object #define GETOBJCMD_F_REPLDB0x20 // 20=Replicate Data Base WORD  wFunction; // Function of commandblock #define GETOBJCMD_M_OPENFLAGS_INC 0x01 // 0x01 = Object open flagsincluded #define GETOBJCMD_M_COMPSLOT_INC 0x02 // 0x02 = CompressionSlot Follows #define GETOBJCMD_M_OBJNAME_INC 0x04 // 0x04 = Object NameFollows #define GETOBJCMD_M_PAYLOAD_INC 0x08 // 0x08 = Payload includedin command #define GETOBJCMD_M_VERINFO_INC 0x10 // 0x10 = VerificationInfo Follows #define GETOBJCMD_M_REPLINFO_INC 0x20 // 0x20 =Verification Info Follows WORD  wMask, // Additional info for commandblock } HAP_GET_OBJECT_COMMAND_BLOCKHDR,*pHAP_GETOBJECT_COMMAND_BLOCKHDR; //Same as Lotus′ TIMEDATE struct, this frees this file from//dependencies on their include files typedef struct { DWORD Innards[2];} HAP_TGTIMEDATE, *pHAP_TGTIMEDATE; typedef struct { HAP_TGTIMEDATETimeDate; DWORD   dFirstNoteID; }HAP_REPLICATE,*pHAP_REPLICATE; // //GET OBJECT COMMAND BLK // typedef struct {HAP_GET_OBJECT_COMMAND_BLOCKHDR Hdr; // Fixed Header MWORD  wOpenFlags;// (if bMask|=0x02) MWORD  wSlot; // Compression slot (if bMask|=0x01)BYTE  sObjectName[1]; // (if bMask|=0x04) Variable Length DWORD dPayloadLen; // (if bMask|=0x04) Length of Payload BYTE  UserData[1];// (if bMask|=0x04) Payload, Variable Length #define GETOBJCMD_VST_32FCS// 1=32 bit FCS of object payload #define GETOBJCMD_VST_LNSEQ 2 //2=Lotus Notes Sequence # #define GETOBJCMD_VST_WLMD 3 // 3=Web LastModified Dates #define GETOBJCMD_VST_WET 4 // 4=Web Entity Tags BYTE bVerifierSignatureType; // (if bMask|=0x08) MWORD wVerifierSignatureLength; // (if bMask|=0x08)  BYTEsVerifierSignature[1]; // (if bMask|=0x08) Signature, Variable LengthHAP_REPLICATE ReplicateInfo; // replication info structure }HAP_GET_OBJECT_COMMAND_BLOCK,*pHAP_GET_OBJECT_COMMAND_BLO CK; // // GETOBJECT RESPONSE HEADER // typedef struct { BYTE  bFunctionCHSe; //always 0xC8 BYTE  bStatus; // DWORD  dMessageLen; WORD wSequenceValue;  // corresponds to original request DWORDdConnectionHandle; // DWORD  dVerifiersDbHandle; // 0 for Web ObjectsBYTE  bProtocol;. BYTE  bResponseBlockCount; // Number of GetObjectresponse blocks // which follow } HAP_GET_OBJECT_RSPHDR,*pHAP_GET_OBJECT_RSPHDR; // // GET OBJECT RESPONSE // typedef struct {HAP_GET_OBJECT_RSPHDR Hdr; // fixed header BYTE  FirstResponseBlock[1];BYTE  sMd5Signature[16];  // if packet signatures required }HAP_GET_OBJECT_RSP, *pHAP_GET_OBJECT_RSP; // // GET OBJECT RESPONSE BLKHEADER // typedef struct { MDWORD dLength; // Length of response block#define GETOBJRSP_F_FIC 1 // 0x01 = The first piece of an object (moreto follow) #define GETOBJRSP_F_MIC 2 // 0x02 = A middle piece of anobject (more to follow) #define GETOBJRSP_F_LIC 4 // 0x04 = The lastpiece of an object #define GETOBJRSP_F_OIC (GETOBJRSP_F_FIC |GETOBJRSP_F_LIC) // 5=Entire Object #define GETOBJRSP_F_VERACK 8 // 0x08= Verification Acknowledgement #define GETOBJRSP_F_GETACK 0x10 // 0x10 =Get Acknowledgement #define GETOBJRSP_F_REPLACK 0x20 // 0x20 =Replication Acknowledgement WORD  wFunction; // Function of response#define GETOBJRSP_M_COMPSLOT_INC0x01 // 0x01 = Compression Slot Follows#define GETOBJRSP_M_OBJNAME_INC 0x02 // 0x02 Object Name Follows #defineGETOBJRSP_M_VERINFO_INC 0x04 // 0x04 = Verification Info Follows #defineGETOBJRSP_M_NAMEIMP 0x08 // 0x08 = Name Implied by previous command#define GETOBJRSP_M_PAYLOAD_INC 0x10 // 0x10 = Payload included #defineGETOBJRSP_M_ERRORLOC 0x20 // 0x20 = Error locating object WORD  wMask;// Additional info for response block }H A P _ G E T _ O B J E C T _ R E S P O N S E _B L O C K H D R ,*pHAP_GET_OBJECT_RESPONSE BLOCKHDR; // // GET OBJECT RESPONSE BLK //typedef struct { HAP_GET_OBJECT_RESPONSE_BLOCKHDR   Hdr; // fixedheader; MWORD  wSlot; // (ifbMask|=0x01) Compression slot BYTE sObjectName[1]; // (if bMask|=0x02) Object Name, Variable Length MDWORDdObjectPayloadLength; // (if bMask|=0x10)  Length of Payload MDWORDdObjectPayUnCompLength; // (if bMask|=0x10) UnCompressed Payload LengthWORD  wCompressionAlgorithm; // (ifbMask|=0x10) Compression AlgorithmBYTE  ObjectData[1]; // (ifbMask|=0x10) Payload, Variable Length MDWORDdErrorStatus; // (if bMask|=0x20) Error status }HAP_GET_OBJECT_RESPONSE_BLOCK,*pHAP_GET_OBJECT_RESPONSE_BLOCK;

In order to generate the signature, each BSG 44 and 56 employ a protocolfor performing a cyclic redundancy check (CRC) on the specified datawhich includes signature and coordinates (an offset into the file andspecifying length of the data). The protocol the CRC employs is a fastmethod for generating a 64 bit CRC on a 32 bit CPU, The existing bitstructure on a 32 bit CPU is that of the type pppfcs 32 algorithmdescribed in RFC1662.TXT by W. Simpson.

The invention modifies the structure as follows: The 64 bit value willconsist of two parts:

1. The existing 32 bit value will be utilized.

2 An additional 32 bits will be derived by dividing the length by fourand performing the operation on four groups of the byte stream. On eachof the four instances the least significant 8 bytes of the “in progress”frame check sequence (a 32 bit value computed by repetitivelyexclusive-oring a constant retrieved by indexing a character stream intoa table of contents) will be appended to a second 32 bit number whichwas initialized to zero.

Modification is as follows:

/*Function*/ void PppFcs64(cp, len, fcsarray) // // Calculate an 8 newFCS given the current FCS and the new, data. // Note! Do not attempt touse this if the length of the // data stream is less than eight bytes.// // ) register u32 fcs; register unsigned char *cp; register int len,remainder; register unsigned char fcsarray[8] { register int 1, lenBy4;fcs 0 LenBy4 = len>> 2; // Divide length by 4 (shift right 2) remainder= len - (LenBy4+LenBy4+LenBy4); fcs = pppfcs32( fcs, cp, LenBy4);fcsarray[4] = (unsigned char) fcs; cp += LenBy4; fcs = pppfcs32( fcs cp,LenBy4 ); fcsarray[5] (unsigned char) fcs; cp += LenBy4; pppfcs32( fcs,cp, LenBy4); fcsarray[6] - (unsigned char) fcs; cp +== LenBy4; fcs =pppfcs32( fcs, cp, Remainder); fcsarray[7] (unsigned char) fcs; *((unsigned long *) fcsarray) ntohl( fcs );

These signatures of data which are generated are placed through thecomparator 58, which, for example, are in this case associated with thecache verifying computer 14. It is recognized that other locations forthe comparator 34 may exist. The comparator; 34 then determines whetherthe signatures of data match. It is recognized that other types ofverification tools may be employed to carry out the present inventionsuch as the MD5 algorithm which is described in RFC1322.TXT by R.Rivest.

In order to generate the Signature of Directory, the DSG 57 is employedby the CVA 53. Referring to FIG. 750, the procedure used to calculatethe Signature of directory is described below:

typedef struct { BYTE bCentury; // 0×13 means nineteen hundred somethingBYTE bYear; // 0×60 means 96 BYTE bMonth; // 0×01 means January, 0×0c =dec. BYTE bDay; // 0×01 through 0×1F BYTE bHour; // 0×00 thru 0×17 BYTEbMinute; // 0×00 thru 0×3B BYTE bSecond; // 0×00 thru 0×3B }NETWORK_TIME, *PNETWORK_TIME; typedef struct { BYTE zFileName[ 32 ]; //null padded NETWORK_TIME sTime; MDWORD dSize; // up to 4 Gig. }DIR_SUBOBJECT;

With respect to the operative steps 750 et seq., the CVA 54 initializes751 the signature to a value of zero. CVA 54 retrieves 752 the firstdirectory sub-object from FS 18 and expresses this as described in theDIR_SUBOBJECT data structure. CVA 54 asks “is this the last directorysub-object?” 753. If the answer is no then CVA 54 factors 754 thesignature of the sub-object into the signature of directory by invokingthe logic CSXorDirEntry below:

void CSXorDirEntry(BYTE *1pSignature,POD_DIR_INFO 1pOdDirInfo)

{

DWORD *1pdSign=(DWORD *)1pSignature;

DWORD *1pdDirInfo=(DWORD *)1pOdDirInfo;

int I;

int j=0;

for (I=0;i<(sizeof(OD_DIR_INFO)/sizeof(DWORD));I++,1pdDirInfo++)

{

1pdSign[j]{circumflex over ( )}=*1pdDirInfo;

j=1−j;

}

}

CVA 54 then retrieves the next directory sub-object from FS 18 andproceeds to 753. If the answer to 753 is yes the then CVA 64 “sends” 756back to NFOC 42 via reverse channel.

The remote client computer 12 is also preferably designed with apredefined portion of its memory 34 operably associated with the NFOC 42for storing a hit ratio @ defined as the percentage of times that a READrequest for a particular data was successfully verified to be in cachememory out of a predetermined number of attempted verifications. Theremote client computer 12 employs HRA 45 as an operative means whichkeeps a record of file names that have been determined to be unworthy ofcaching because previously determined hit ratios fell below thepredetermined threshold and, thus, future accesses of the data to suchfiles will be inhibited from being subject to the processes definedherein. In other words, the HRA 45 dumps data from cache memory if thedata associated hit ratio was less than a predetermined number,otherwise the data remains in cache memory.

Still, another embodiment of the invention is set forth hereinafter. Theexemplary code sets forth operations of determination of whether thedata objects are to be transferred via the WAN or LAN, initiatingcompressing/decompressing the data objects upon determining the mode oftransfer, i.e., through the WAN or LAN, and assembling/disassembling thedata objects into a stream.

// // Prime Replication REQUEST // typedef struct { BYTE  bFunctionCHSe; // always 0x4e BYTE  bResv;    // DWORD  dMessageLen; WORD  wSequenceValue;   // DWORD  dConnectionHandle;  // DWORD  dVerifiersDbHandle;  //0 for Web Objects BYTE   ReplList[1]; }HAP_PRIME_REPL_REQ,*pHAP_PRIME_REPL_REQ; // // Prime ReplicationRESPONSE // typedef struct { BYTE  bFunctionCHSe;   // always 0x4e BYTE bStatus;    // DWORD  dMessageLen; WORD wSequenceValue;   // DWORD dConnectionHandle;  // DWORD  dVerifiersDbHandle;  // 0 for Web ObjectsWORD  wReplStatus; } HAP_PRIME_REPL_RSP,*pHAP_PRIME_REPL_RSP; // //Starting Replication Response - not used right now // typedef struct {BYTE  bFunctionCHSe;   // always 0x4B BYTE  bStatus;    // DWORD dMessageLen; WORD  wSequenceValue;   // Same value as in request DWORDdVerifiersDbHandle;  // WORD   wReplStatus; DWORD  dNoteCount;      //Number of notes which will be retrieved for this replicate }HAP_START_REPL_RSP,*pHAP_START_REPL_RSP; // // STOP SENDING REQUEST //typedef struct { BYTE  bFunctionCHSe;   // always 0x49 BYTE  bResv;   // DWORD dMessageLen; WORD  wSequenceValue;   // DWORD dConnectionHandle;  // DWORD  dVerifiersDbHandle;  // 0 for Web ObjectsDWORD  dReason; } HAP_STOP_SENDING_REQ,*pHAP_STOP_SENDING_REQ; // //FLUSH REQUEST // typedef struct { BYTE  bFunctionCHSe;   // always 0x49BYTE  bResv;    // DWORD  dMessageLen; } HAP _FLUSH_REQ,*pHAP_FLUSH_REQ;// // FLUSH RESPONSE // typedef struct { BYTE  bFunctionCHSe;   //always 0x49 BYTE  bResv;    // DWORD dMessageLen; WORD  wSequenceValue;  // DWORD  dConnectionHandle;  // DWORD  dVerifiersDbHandle;  // 0 forWeb Objects } HAP_FLUSH_RSP,*pHAP_FLUSH_RSP; // // PUT OBJECT REQUESTHEADER // typedef struct { BYTE  bFunctionCHSe;   // always 0x4f BYTE bResv;    // DWORD dMessageLen; WORD  wSequenceValue;   // DWORD dConnectionHandle;  // DWORD  dVerifiersDbHandle;  // 0 for Web ObjectsBYTE  bProtocol; BYTE  bCommandBlockCount;  // Number of GetObject //command blocks which follow }HAP_PUT_OBJECT_REQHDR,*pHAP_PUT_OBJECT_REQHDR; // // PUT OBJECT REQUEST// typedef struct { HAP_PUT_OBJECT_REQHDR Hdr;  // fixed header BYTE FirstCommandBlock[1]; } HAP_PUT_OBJECT_REQ,*pHAP_PUT_OBJECT_REQ; // //PUT OBJECT COMMAND BLK HEADER // typedef struct { MDWORD dLength; //Length of command block #define PUTOBJCMD_F_PUTOBJECT 0x08  // 08= Sendthis Object to verifier #define PUTOBJCMD_F_PUSHREPLDB 0x10  // 10=PushReplicate start request #define PUTOBJCMD_F_PUSHREPLDBCMPLT 0x20  // 20=Push Replicate cmplt request WORD wFunction; // Function of commandblock WORD wMask; // Additional info for command block }HAP_PUT_OBJECT_COMMAND_BLOCKHDR,*pHAP_PUT_(—) OBJECT_COMMAND_BLOCKHDR;// // PUT OBJECT COMMAND BLK // typedef struct {HAP_PUT_OBJECT_COMMAND_(—) // Fixed Header BLOCKHDR Hdr; MWORD wSlot; //Compression slot (if bMask|=0x01) BYTE  sObjectName[1]; // (ifbMask|=0x04) Variable Length DWORD  dErrorStatus; MDWORDdObjectPayloadLength; // (if bMask|=0x10) Length of Payload MDWORDdObjectPayUnCompLength; // (if bMask|=0x10) UnCompressed Length ofPayload MWORD wCompressionAlgorithm; // (if bMask|=0x10) CompressionAlgorithm BYTE   ObjectData[1]; // (if bMask|=0x10) Payload, VariableLength } HAP_PUT_OBJECT_COMMAND_BLOCK,*pHAP_PUT_OBJECT COMMAND_BLOCK; //// PUT OBJECT RESPONSE HEADER // typedef struct { BYTE  bFunctionCHSe;  // always 0xC8 BYTE  bStatus;    // DWORD  dMessageLen; WORD wSequenceValue;   // corresponds to original request DWORD dConnectionHandle;  // DWORD  dVerifiersDbHandle;  // 0 for Web ObjectsBYTE  bProtocol; BYTE  bResponseBlockCount; // Number of GetObjectresponse // blocks which follow }HAP_PUT_OBJECT_RSPHDR,*pHAP_PUT_OBJECT_RSPHDR; // // PUT OBJECT RESPONSE// typedef struct { HAP_PUT_OBJECT_RSPHDR Hdr;  // fixed header BYTE FirstResponseBlock[1]; BYTE  sMd5Signature[ 16 ]; // if packetsignatures required } HAP_PUT_OBJECT_RSP,*pHAP_PUT_OBJECT_RSP; // // PUTOBJECT RESPONSE BLK HEADER // typedef struct { MDWORD dLength;  //Length of response block WORD  wFunction;  // Function of response WORD wMask;   // Additional info for response block }HAP_PUT_OBJECT_RESPONSE_BLOCKHDR,*pHAP_PUT_(—) OBJECT_RESPONSE_BLOCKHDR;// // PUT OBJECT RESPONSE BLK // typedef struct {HAP_PUT_OBJECT_RESPONSE_BLOCKHDR // fixed header Hdr; BYTE sObjectName[1]; // (if bMask|=0x02) Object Name, Variable Length MDWORDdObjectPayloadLength; // (if bMask|=0x04) Length of Payload MDWORDdObjectPayUnCompLength; // (if bMask|=0x04) UnCompressed Length ofPayload WORD wCompressionAlgorithm; // (if bMask|=0x04) CompressionAlgorithm BYTE ObjectData[1]; // (if bMask|=0x04) Payload, VariableLength MDWORD dErrorStatus; // (if bMask|=0x01) Error status WORD  wNumUpdateErrors; } HAP_OBJECT_RESPONSE_BLOCK*pHAP_PUT_OBJECT_(—)RESPONSE_BLOCK;

While the preferred embodiment has been set forth above, it is done soonly by way of example and not intended to be limiting to the scope ofthe claims appended hereto. It is believed that modifications andvariations of the present invention will be readily apparent to thoseskilled in the ail will be coming within the scope of the claims hereto.

What is claimed is:
 1. A file/object server CPU having which utilizes data/object compression, which includes: a compression device having means for determining when processing said CPU reaches a predetermined percentage of maximum processing capacity, and means operably associated with said determining means for controlling compression of data/object upon reaching said predetermined percentage in a manner to aid processing to fall below said predetermined percentage.
 2. The file/object server CPU of claim 1, wherein said controlling means enables compression of data/object upon falling below said predetermined percentage.
 3. The file/object server CPU of claim 1, wherein said determining means is further characterized to monitor the processing capacity over time.
 4. The file/object server CPU of claim 1, wherein said determining means is further characterized to determine average processor usage over a period of time for previously processed data/object.
 5. The file/object server CPU of claim 4, wherein said previously processed data/object is of a size approximating a size of a to be processed data/object.
 6. The file/object server CPU of claim 1, wherein said determining means is characterized to include means for evaluating average process usage percentage of data/object to be compressed prior to compression.
 7. The file/object server CPU of claim 1, which is further characterized to include: a file/object server computer having an operating system, a first memory a permanent storage memory, and a processor; a remote client computer operably connected to said file/object server computer in a manner to rapidly transfer data objects, having an operating system, a first memory, a permanent storage memory, and a processor; a communication link operably connecting said remote client computer and said file/object server computer including means for routing between a WAN and a LAN; and means operably associated with one of said file/object server computer and said remote client computer for determining whether data objects are transferred through said WAN or said LAN; means for compressing said data/objects upon detecting transfer through said WAN; and means for assembling said data/objects into a stream and transferring said data objects through one of said WAN and said LAN.
 8. The invention in accordance with claim 7, which includes means operably associated with one of said file/object server computer and said remote client computer for disassembling and decompressing said assembled data objects into individual data objects.
 9. The invention in accordance with claim 7, which further includes means operably associated with one of said file/object server computer and said remote client computer for determining a list of objects to be transferred during a replication/synchronization process.
 10. A method for increasing data access from data of the type including at least one of a file, an object and a directory in a file/object oriented network, which comprises: employing a file/object server computer having an operating system, a first memory, a permanent storage memory, and a processor, and which includes a compression device having means for determining when processing said server computer reaches a predetermined percentage of maximum processing capacity, and means operably associated with said determining means for controlling compression of data/object upon reaching said predetermined percentage in a manner to aid processing to fall below said predetermined percentage; employing means for compressing said data/object upon detecting transfer through said WAN; and employing means for assembling said data objects into a stream and transferring said data/ object through one of said WAN and said LAN.
 11. The method of claim 10, which further includes: employing a remote client computer operably connected to said file/object server computer in a manner to rapidly transfer data objects, having an operating system, a first memory, a permanent storage memory, and a processor; employing a communication link operably connecting said remote client computer and said file/object server computer including means for routing between a WAN and a LAN; and employing means operably associated with one of said file/object server computer and said remote client computer for determining whether data objects are transferred through said WAN or said LAN.
 12. The method of claim 10, which includes employing means operably associated with one of said file/object server computer and said remote client computer for disassembling and decompressing said assembled data objects into individual data objects.
 13. The method of claim 10, which further includes employing means operably associated with one of said file/object server computer and said remote client computer for determining a list of objects to be transferred during a replication/synchronization process. 